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Abstract 

This paper provides a simple proof of the mutual incoherence condition {p < 2k-i ^ under which 
/C-sparse signal can be accurately reconstructed from a small number of linear measurements using the 
orthogonal matching pursuit (OMP) algorithm. Our proof, based on mathematical induction, is built on 
an observation that the general step of the OMP process is in essence same as the initial step since the 
residual is considered as a new measurement preserving the sparsity level of an input vector. 

Index Terms 

Compressive sensing, orthogonal matching pursuit (OMP), restricted isometric property (RIP), 
mutual incoherence condition. 



This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) 
(No. 2010-0012525) and the research grant from the second BK21 project. 



January 22, 2013 



DRAFT 



1 

A Simple Proof of the Mutual Incoherence 
Condition for Orthogonal Matching Pursuit 



I. Introduction 

As a sampling paradigm guaranteeing the reconstruction of sparse signal with sampling rate 
significantly lower than the Nyquist rate, compressive sensing (CS) has received considerable 
attention in recent years JD, [IS, The main goal of the CS is to accurately reconstruct a 
high dimensional sparse vector using a small number linear measurements. Specifically, for a 
given matrix $ G M™^" (n > m), the CS recovery algorithm generates an estimate of fC-sparse 
vector X G M" from a set of linear measurements 

y = ^x. (1) 

Although this task seems to be a severely ill-posed inverse problem, due to the prior knowledge of 
sparsity information, x can be perfectly reconstructed via properly designed recovery algorithm. 
Among many greedy search algorithms developed for this purpose, OMP algorithm has received 
special attention due to its simplicity and competitive reconstruction performance [H]. 

Theoretical analysis of OMP to date has concentrated primarily on two fronts. The first 
approach is based on the restricted isometric property (RIP). A sensing matrix $ satisfies the 
RIP of order K if there exists a constant S such that [|71 

(l-5)||x||^<||$x||^<(l + 5)||x||5 (2) 

for any /C-sparse vector x (||x||q < K). In particular, the minimum of all constants 5 satisfying 
(O is called the isometry constant 5k- Wakin and Davenport have shown that the OMP can 
reconstruct all K-sparse signals if 6k+i < This result has been recently improved by 

Wang and Shim to 6k+i < f^l- The second approach is based on the coherence parameter. 

The coherence parameter n of the sensing matrix $ is defined as 

/i = max \{(pi,(pj)\ 

where (fi and (^j are two column vectors of When the columns of ^> have unit norm and 
satisfy the mutual incoherence condition given by < 2k-i ^ ^he OMP will recover /^-sparse 
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TABLE I 
OMP Algorithm 



Input: 

Initialize: 

While 



End 

Output: 



y, K. 

fc = 0, r" = y, T" = 0. 
k<K 
k^k + 1. 

(Identify) t'^ = argmaxj \ {r''~^,ipj) 
(Augment) T'' = T^'^ U {t^}. 
(Estimate) x^k — argmin ||y — $2-^x112. 
(Update) r'"' = y — ^>yfcXyfc. 



X = arg min lly — $x|U. 

^x:supp(x)=T^< " "2 



signal X from the measurements y = $x [HI. It is well known that this result is also applied to 
^1 -minimization approach [[8l. 

In this work, we provide a simple proof of the mutual incoherence condition for the OMP 
using mathematical induction. Our proof is built on an observation that the general step of the 
OMP process is in essence same as the initial step since the residual is considered as a new 
measurement preserving the sparsity level of an input vector. The mutual incoherence condition 
for the OMP is formally described in the following theorem. 



Theorem 1 (Mutual incoherence condition for OMP): For any f^-sparse vector x, the OMP 
algorithm perfectly recovers x from the measurements y = $x if the coherence parameter n 
satisfies 



II. Simple Proof of Theorem [H 
Before presenting the proof of Theorem [1] we provide lemmas useful in our analysis. 
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Lemma 2 (Norm inequality For A and B in R'"^", a G M, and x G M", following 
inequalities are satisfied: 

IIAII2 < A/mri||A||max, 

IIA + BII2 < IIAII2 + IIBII2, 
IIABII2 < IIAII2IIBII2, 

||aA||2 = \a\ II AII2, 
||Ax||2 < ||A||2||x||2, 

where ||A||2 is the spectral norm of A and ||A||inax is the maximum absolute value of elements 
of A (i.e., ||A||max = maxij |aij|). 

Lemma 3 (A direct consequence of RIP Let / C {1, 2, ■ ■ ■ , n} and $7 be the restriction 
of the columns of $ to a support set /. If 5\i\ < 1, then for any u G M'^', 

(1 - ||U||2 < Wj^iuW^ < (1 + 5\i\) ||U||2. 

Lemma 4: The isometric constant 5k for the sensing matrix $ satisfies 

5K<{K-l)^l. 

Proof: 

Using Lemma [2l we have 

||$rxr||2 = \\'^t^'t^t^t\\2 

< l|Xrll2ll*T*T|l2l|xT||2 

= II^t^tIUIxtIIs 



(4) 



where T is the support of x (a set with the locations of the non-zero elements of x), xy is a 
vector composed of the elements of x indexed by T. Noting that (i, j)-th element of is 
((^i, i^j) and is the unit norm vector, it is clear that 

/ 1 (V51,V22) ■■■ (v?l,V5n)\ 



(5) 
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Now, let 



then 



A 



$^$T = (1 - + A 



(6) 



(7) 



and thus HAjlj^^g^^ = /x. Hence, 

Wr^Th = 1I(1-/^)I + A||2 

< ||(l-/x)I||2 + ||A||2 

< l-M + v^i|A||_ 
= 1 + {K-I)fi 

where ([10] ) is from Lemma |2] Using (H) and (fTTT ). we have 

||^TXt||2 < II*T*t||2 I|xt||2 

< (l+(ir-l)/i)||XT||^ 

Recalling the definition of the RIP that 5k is the minimum satisfying we have 



(8) 
(9) 
(10) 

(11) 



Proof of theorem [T] 

Proof: We will prove the theorem using induction. In the first iteration (k = 1) of the OMP 
algorithm, t^(= t^) becomes the index of the column maximally correlated with the measurement 
y, i.e.. 



argmax|(v?i,y) 



(12) 
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Then, we have 



\{^t^,y)\ = max|((/?i,y)| (13) 

> ^myh (14) 



> ^||$'r$TXT||2 (15) 

V K 



> -^(1-5k)M2 (16) 



> -l=(l-(K-l)fi)\\^Th (17) 



where (fTST ) is due to y = ^^x^-, (fT6l) and (flTl) follow from Lemma |3] and HI respectively. 
Now, suppose that t'' is not belonging to the support of x (i.e., t'' ^ T), then 

\{^tk,y)\ = \\(p'tk^T^T\\2 (18) 

< y^.^rU^Th (19) 

/5^|(^,.,^,)n|xT||2 (20) 



< jE-^'ii^^ii^ (21) 



Ki^W^rh (22) 

where (|2TI) is from the definition of /i (/i = max | (y^j, yjj) |). This case, however, will never occur 
if 

^ (1 - (ir - 1) /i) llxrL > VKfi\M\2 (23) 

or 

^ < (24) 

In summary, if fi < 2k-v ^^^^ t'' E T for the first iteration of the OMP algorithm. 

Now we assume that the former k iterations are successful (T'' = ■ ■ ■ G T) for 

1 < k < K -1. Then it suffices to show that t''+^ is in T but not in {t^^^ G T\T^). Recall 
from Table H] that the residual at the k-ih. iteration of the OMP is 

r'^ = y - ^^fcx^fc. (25) 



January 22, 2013 



DRAFT 



6 



Since y = ^t^t and ^rpk is a submatrix of G span ($t) and thus r'^ can be expressed 

as a linear combination of the |T| (= K) columns of Accordingly, we can express r'^ as 
r'^ = $x' where the support (set of indices for nonzero elements) of x' is contained in the support 
of X. In this sense, it is natural to interpret that r*^ is a measurement of i^-sparse signal x' using 
the sensing matrix Thus, if (|24|) is satisfied, we guarantee that G T at the (k + l)-th 
iteration. Noting that the residual r*^ is orthogonal to the columns already selectee]!, index of 
these columns is not selected again (see the identify step in Table I) and hence G T\T''. 
This concludes the proof. ■ 



Thus far, we have shown that the OMP algorithm is working perfectly if the sensing matrix 
$ satisfies the condition /i < 2k-i - Interestingly, this condition is not only sufficient but also 
necessary. We prove this claim by showing that, even with slight relaxation of this condition 
(/i = 27r^)' it is possible that the OMP algorithm cannot perfectly recover i^-sparse signal. 
Note that our construction of $ is similar to Cai, Wang, and Xu's work for proving the tightness 
of mutual incoherence condition for £i -minimization [9, Remark 3.2]. 

Remark 1 (Necessity of jj, < jw^)-' Suppose ^» has normalized columns (Hv^iHa = f'ifi = 1) 
and also satisfies /i = . Then it is clear that G M"^" has a unit diagonal and the 



absolute value of the off-diagonal elements is upper bounded by 577— r. Now consider 



1 



1 



2K-1 



2K-1 
1 



2K-1 
1 

' 2K-1 



J 



\ 2K-1 2K-1 

Then is symmetric and positive semi-definite matrix, and hence $ can be found by an 
eigen-decomposition of f9l. Note that an n x n matrix K with (K)j ^ = a and (K)i,j = b 
for i ^ j is invertible if and only if a + (n — l)b ^ 0. Hence, for a = 1 and b = — 2k-i ^ 
is not invertible for the choice of n = 2K. In this case, eigen-decomposition of becomes 



= UAU' 



(26) 



'since ^ytXyt is a projection of y, the error vector y — #yfcXyfc (which equals r*") is orthogonal to the projection ^^fcXyt 
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and 



where 



A 



Ai 
A2 





* = Vau' 

■ 

• 

• A, 













v 



2K-1 



J 



and {\i}i=i,2,- ,1 are / nonzero eigenvalues of Since the rank of $ is / (< 2K), there exists 



a vector z G 



b2K 



(which by definition is 2i^r-sparse vector) in the null space of # obeying 



$z — 0. One can then divide z into two X-sparse vectors Xi and — X2 (i.e., z = Xi — X2). 
This gives $xi = $X2 so that the OMP algorithm fails to recover X-sparse vector. In fact, no 
reconstruction algorithm can always guarantee the perfect recovery of X-sparse vector under 
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